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EXECUTIVE  SUMMARY 

A .  PURPOSE 

The  purpose  of  this  thesis  is  to  compare  and  analyze  the  effects  of 
the  Janus (A)  default  terrain  database  and  a  Janus (A)  modified  terrain 
database  on  a  modeled  Javelin  operational  test.  It  is  part  of  the  Model- 
Test  -Model  (M-T-M)  research  program  sponsored  by  the  U.S.  Army  Training  and 
Doctrine  Command  (TRADOC) ,  Analysis  Command -Monterey  (TRAC-MTRY)  and  the 
Test  and  Experimentation  Command  (TEXCOM) ,  Experimentation  Center  (TEC) . 
This  is  part  of  an  ongoing  effort  to  accredit  the  M-T-M  concept. 

The  research  for  this  thesis  was  accomplished  in  three  phases.  First, 
the  operational  test  area  was  defined  and  two  duplicate  Janus (A)  terrain 
database  files  were  created.  The  terrain  characteristics  of  one  of  the 
files  were  modified  so  that  the  vegetation  information  per  50  meter  grid 
cell  more  accurately  represented  that  of  the  actual  test  site.  Next,  a 
data  analysis  comparison  was  done  on  the  two  files  to  determine  if  their 
distributions  are  the  same.  Lastly,  the  Javelin  test  scenarios  were  run 
using  both  terrain  files  and  the  results  compared  using  statistical 
procedures . 

B.  BACKGROUND 

With  projected  budget  cuts,  fewer  defense  dollars  will  be  available 
for  operational  test  and  evaluation  (OT&E)  of  weapon  systems.  The  M-T-M 
concept  integrates  operational  field  tests  with  simulation  models  to 
increase  the  effectiveness  and  efficiency  of  operational  tests.  The 
concept  of  M-T-M  is  to  use  high  resolution  simulation  model (s)  before  the 
operational  test  to  help  design  efficient  and  meaningful  test  scenarios  and 
tactics,  and  after  the  field  test  to  expand  the  evaluation  effort  to 
include  testing  other  scenarios  and  tactics.  One  of  the  models  currently 
in  use  is  Janus (A) ,  a  computer-based,  two-sided,  high  resolution, 
stochastic  combat  simulation  model  designed  for  Army  training  and  combat 
development.   It  uses  a  digitized  terrain  database  to  represent  a  portion 


vm 


of  the  real  terrain.   This  database  is  based  on  Digital  Terrain  Elevation 
Data  (DTED)  Level  I  data  provided  by  the  Defense  Mapping  Agency. 

C .  RESULTS 

The  results  of  the  analysis  indicated  that  vegetation  code 
distributions  of  the  default  terrain  database  and  the  modified  terrain 
database  were  not  the  same.  Three  scenarios,  a  deliberate  night  defense, 
a  deliberate  day  defense  and  a  hasty  day  defense  were  modeled  using  each 
of  the  terrain  databases.  Data  was  collected  using  the  detection  range  of 
an  enemy  vehicle  by  the  opposing  force  as  the  measure  of  effectiveness. 
The  three  different  scenarios  were  each  run  eight  times  on  each  terrain 
database.  Analysis  indicated  that  there  was  a  statistically  significant 
difference  between  the  detection  ranges  for  two  of  the  scenarios.  These 
two  scenarios  were  run  on  terrain  that  required  extensive  modification  to 
the  vegetation  codes.  No  significant  difference  was  noted  in  the  third 
scenario.  Very  few  modifications  of  the  terrain  data  were  needed  in  the 
area  where  this  battle  was  fought. 

D.  RECOMMENDATIONS 

The  results  do  not  suggest  that  the  Janus (A)  model  is  invalid. 
However,  the  results  do  indicate  that  further  improvements  to  the  terrain 
database  will  further  reduce  the  errors  between  results  based  on  the 
simulated  terrain  and  the  actual  terrain.  This  will  result  in  better 
simulation  of  the  line  of  sight,  and  the  effectiveness  of  a  weapon  system 
for  a  given  scenario  and  tactic.  Thus,  the  overall  results  of  the  model 
should  better  reflect  the  operational  test  results. 
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I .       INTRODUCTION 

A.  RESEARCH  OBJECTIVES 

This  thesis  is  part  of  the  Model -Test -Model  (M-T-M)  research  program 
sponsored  by  the  U.S.  Army  Training  and  Doctrine  Command  (TRADOC) ,  Analysis 
Command-Monterey  (TRAC-MTRY)  and  the  Test  and  Experimentation  Command 
(TEXCOM) ,  Experimentation  Center  (TEC).  The  M-T-M  concept  integrates 
operational  field  tests  results  with  simulation  models.  As  part  of  an 
ongoing  effort  to  accredit  the  M-T-M  concept,  this  thesis  will  compare 
Janus (A)  model  terrain  to  the  actual  test  site  terrain  for  the  Javelin 
antitank  weapon  system. 

B.  BACKGROUND 

With  projected  budget  cuts,  fewer  defense  dollars  will  be  available 
for  operational  test  and  evaluation  (OT&E)  of  weapon  systems.  The  M-T-M 
concept  integrates  operational  field  tests  with  simulation  models  to 
increase  the  effectiveness  and  efficiency  of  operational  tests.  The 
concept  of  M-T-M  is  to  use  high  resolution  simulation  model (s)  before  the 
operational  test  to  help  design  efficient  and  meaningful  test  scenarios  and 
tactics,  and  after  the  field  test  to  expand  the  evaluation  effort  to 
include  testing  other  scenarios  and  tactics.  Thus,  the  operational  test 
and  evaluation  (OT&E)  community  will  work  with  analysts  using  simulations 
and  models  to  design  operational  tests  and  then  use  these  models  to  make 
predictions  on  the  employment,  manning  level,  supply  rate,  etc.  of  the 
weapon  system.  One  of  the  models  currently  in  use  is  Janus (A) ,  a  computer- 
based,  two-sided,  high  resolution  combat  simulation  model  designed  for  Army 
training  uses  and  combat  development.  In  October  1990,  Mr.  Walter  W. 
Mollis,  Deputy  Under  Secretary  of  the  Army  (Operations  Research)  tasked  TEC 
at  Fort  Hunter  Liggett,  California  to  improve  the  M-T-M  methodology  [HOLL 
89].   TEC  enlisted  TRAC-MTRY  to  conduct  research  in  support  of  the  M-T-M 


concept.  TRAC-MTRY's  research  efforts  are  directed  toward  the  validation 
and  accreditation  of  the  M-T-M  concept  using  the  Janus (A)  high  resolution 
combat  simulation  model. 

The  model  terrain  in  Janus (A)  is  a  digital  representation  of  the 
actual  test  site  terrain  located  at  Fort  Hunter  Liggett.  Extensive  efforts 
have  been  made  to  simulate  the  test  site  terrain  as  closely  as  possible 
because  terrain  characteristics  affect  the  movement  rates  and,  most 
importantly,  the  line  of  sight  (LOS)  of  combat  systems.  If  the  combat 
simulation  model  terrain  accurately  represents  the  test  site  terrain,  one 
can  expect  a  more  realistic  simulation  of  the  LOS  and  the  effectiveness  of 
a  weapon  system  for  a  given  scenario  and  tactic.  Also,  the  weapon  system's 
tactical  emplacement  can  more  closely  reflect  actual  field  locations  which 
affect  both  range  and  LOS.  Thus,  the  overall  results  of  the  model  should 
better  reflect  the  operational  test  results  if  the  errors  between  the  model 
terrain  and  the  actual  terrain  are  reduced. 

C.    RESEARCH  METHODOLOGY 

The  research  for  this  thesis  was  accomplished  in  three  phases.  First, 
the  operational  test  area  was  defined  and  two  duplicate  Janus (A)  terrain 
database  files  were  created.  The  vegetation  codes  in  one  of  the  files  were 
modified  to  more  accurately  represent  the  actual  test  site  terrain.  To 
help  ensure  accuracy,  two  separate  terrain  walks  were  conducted  to  study 
the  vegetation  and  determine  densities  for  various  tree  stands  from 
different  locations  in  the  test  site  area.  Information  was  also  gained 
from  an  eight  meter  resolution  terrain  database  extracted  from  the 
Perspective  View  Generator  and  Analysis  Systems  for  Unmanned  Sensors 
(PEGASUS)  replicator.  The  PEGASUS  terrain  database  creation  system  will 
be  explained  in  a  later  chapter.  Next,  a  data  analysis  comparison  was  done 
on  the  vegetation  code  distributions  from  an  identical  one  kilometer  area 
in  each  of  the  two  files  to  determine  their  similarities  and  differences. 
Third,   the  Javelin  test  scenarios  were  run  using  both  terrain  files  and 


the  results  compared  using  statistical  procedures.  The  measure  of 
effectiveness  (MOE)  used  to  compare  the  results  was  the  detection  range  of 
an  enemy  vehicle  by  the  opposing  force. 

D.    ORGANIZATION 

This  thesis  is  organized  in  five  chapters.  Chapter  II  describes  the 
Javelin  and  the  Model -Test -Model  concept.  Chapter  III  contains  a 
description  of  Janus (A) ,  the  PEGASUS  replicator,  the  process  required  to 
build  a  terrain  database  and  the  methods  used  to  perform  terrain 
modifications.  Chapter  IV  contains  the  discussion  and  results  of  the 
vegetation  code  analysis  and  the  Javelin  scenario  runs.  A  summary  of  the 
findings  and  recommendations  is  given  in  the  last  chapter. 


II.   JAVELIN  AND  THE  MODEL-TEST-MODEL  CONCEPT 

A .    JAVELIN 

The  success  of  the  U.S.  in  the  Persian  Gulf  War  has  prompted  potential 
hostile  countries  to  upgrade  their  military  forces  with  more  advanced 
weapon  systems.  Thus,  the  U.S.  military  can  expect  to  face  enemy  armored 
vehicles  with  considerable  improvements  in  lethality,  mobility  and 
survivability.  To  meet  this  new  threat,  the  U.S.  has  had  to  continue  to 
improve  and  field  new  ant i -armor  weapon  systems,  capable  of  defeating  these 
new  threats.  Currently,  the  U.S.  forces  rely  on  three  antitank  weapon 
systems:  the  tubular  launched  optically  sighted  wire  guided  missile  (TOW), 
the  Dragon  and  the  antitank  4  missile  (AT4).  The  TOW  is  a  wire  guided 
antitank  weapon  capable  of  destroying  enemy  armored  vehicles  at  up  to  3750 
meters.  The  AT4  is  a  hand  held,  fire  and  forget  weapon,  capable  of  killing 
thin  skinned  armored  vehicles  at  ranges  from  65  to  3  00  meters.  The  AT4 
replaced  the  light  anti-armor  weapon  (LAW)  to  give  the  infantryman  more 
firepower  to  defeat  the  new  enemy  armored  personnel  carriers.  The  Dragon 
is  a  medium  antitank  wire  guided  missile  capable  of  destroying  enemy  armor 
out  to  1000  meters. 

However,  with  the  new  advances  in  enemy  armor,  the  U.S.  has  developed 

a  new  more  effective  medium  antitank  missile,  the  Javelin,  to  replace  the 

Dragon. 

The  Javelin  is  a  top  attack,  man  portable  antitank 
missile  which  gives  the  infantryman  an  increased 
capability  to  engage  and  defeat  threat  tanks  and  other 
armored  vehicles.  It  is  a  fire  and  forget  weapon  system 
which  significantly  increases  gunner  survivability 
because  it  no  longer  requires  a  gunner  to  track  the 
target  for  the  duration  of  the  missile's  flight.  [JAVE 
92] 

The  Javelin's  greater  lethality  comes  from  its  ability  to  attack  its 

targets  from  the  top  where  the  armor  is  the  thinnest,  its  2000  meter  plus 


range  and  its  improved  warhead.  The  Javelin  consists  of  three  components, 
the  command  launch  unit  (CLU),  the  launcher  and  the  missile.  Figure  1. 


Figure  1  Javelin  Components 


The  heart  of  the  Javelin  is  the  CLU.  It  is  an  integrated  day/night  sight 
unit  that  uses  conventional  optics  or  thermal  imagery  to  paint  a  picture 
of  the  target  to  the  misile's  memory  allowing  it  to  track  the  target  on  the 
battlefield.  The  Javelin  has  a  rate  of  fire  of  three  rounds  per  minute. 
These  improvements  compared  to  the  DRAGON'S,  1000  meter  range,  rate  of  fire 
(two  rounds  per  minute) ,  wire  guided  missile  and  inability  to  attack  the 
most  vulnerable  parts  of  enemy  armored  vehicles  give  the  dismounted  company 
team  greater  lethality  and  increased  survivability.  Introduction  of  the 
Javelin  to  the  battlefield  will  better  equip  U.S.  forces  to  meet  and  defeat 
new  threat  armor  technology. 

B.    MODEL-TEST-MODEL  CONCEPT 

The  overall  goal  of  the  Model -Test -Model  concept  (M-T-M)  is  to  improve 
operational   test   effectiveness.    M-T-M  accomplishes   this  by  using 


simulation  models  from  which  the  operations  analyst  attempts  to  predict 
operational  test  data  for  a  particular  test  scenario.  The  simulation  model 
provides  insights  into  the  feasibility  of  test  scenarios  and  test 
objectives.  This  information  is  used  by  operational  testers  at  the 
Operational  Test  and  Evaluation  Command  (OPTEC)  to  make  a  more  efficient 
test  design  and  by  the  program  manager  to  help  him  make  more  accurate 
decisions  affecting  the  acquisition  process.  Application  of  M-T-M  includes 
a  force  on  force,  free  play  field  test  that  requires  cooperation  and 
coordination  between  testing  and  analysis/modeling  communities.  This  means 
that  the  modelers  and  analysts  must  be  intricately  involved  with  the 
planning  and  designing  of  tests  and  the  data  collection  and  data  reduction 
techniques.  This  coordination  will  ensure  that  all  parties  involved 
understand  and  account  for  all  the  peculiarities  and  problems  associated 
with  the  operational  test. 

There  are  five  phases  to  M-T-M:  long  term  planning,  pretest  modeling, 
field  test,  post-test  modeling  and  accreditation.  Upon  accreditation  of 
the  model,   it  can  be  used  by  military  agencies  to  conduct  careful 

extrapolation  to  other  scenarios. 

1.  Phase  0  (Long  Term  Planning) 

Phase  0  begins  with  the  analyst  and  tester  defining 
responsibilities.  During  this  phase,  a  Memorandum  of  Agreement  (MOA)  and 
a  Project  Coordination  Sheet  are  signed  by  the  organizations  involved  to 
identify  working  relationships,  specific  roles  and  what  type  of  product  is 

expected. 

2.  Phase  I  (Pretest  Modeling) 

This  is  a  modeling  phase  that  relies  on  high  resolution  combat 
simulation  models  to  aid  in  planning  the  operational  field  test.  The  goal 
is  to  save  time,  money  and  resources  in  the  test  design.  In  this  phase, 
the  model  is  prepared  and  executed  with  the  proposed  scenarios.  The 
scenarios  should  incorporate  the  restrictions  developed  by  the  test 
personnel   (e.g.,   size/characteristics   of   the   terrain   box,   weapon 


capabilities,  rates  of  advance,  force  size,  etc.)-  The  scenario 
restrictions  and  the  tactics  for  each  scenario  are  given  to  the  weapon 
system  proponent  to  provide  the  proponent  subject  matter  experts  (SME)  a 
doctrinal  review  of  the  tactics  and  test  design.  After  the  model  is  run, 
the  test  design  and  tactics  are  refined  as  necessary.  The  results  from  the 
model  are  used  by  the  test  designers  to  focus  data  collection  efforts, 
identify  external  and  internal  constraints  that  will  have  a  negative  impact 
on  the  test,  and  minimize  the  possibility  that  test  objectives  will  not  be 
accomplished.   Phase  I  continues  until  the  model  predicts  that  the  test 

objectives  will  be  met. 

3.  Phase  II  (Field  Test) 

During  this  phase,  the  operational  effectiveness  and  weapon 
system  capabilities  are  evaluated.  The  first  part  of  this  phase  is  the 
Initial  Operational  Test  and  Evaluation  (IOTE).  In  the  IOTE,  maneuver 
units  perform  the  successful  battle  tactics  developed  by  their  unit  leaders 
in  the  pretest  modeling  phase.  The  tactics  are  revised  as  appropriate, 
based  on  the  outcomes  of  the  IOTE.  The  Operational  Test  (OT)  tactics  must 
be  unscripted  to  increase  the  validity  of  the  OT,  but  the  tactics  must 
remain  within  the  test  restrictions.  It  is  crucial  that  the  modeler 
understand  how  the  test  was  conducted  and  works  with  the  data  reduction 
personnel  to  understand  how  the  data  was  collected  and  screened.  At  the 
conclusion  of  the  field  test  phase,  the  modeler  obtains  the  necessary  data 

and  begins  the  post-test  modeling  phase. 

4.  Phase  III  (Post -Test  Modeling) 

In  this  phase,  the  model  is  used  to  provide  analysis  and  feedback 
to  test  personnel,  to  explain  unexpected  test  results  and  to  guide  the 
modifications  of  further  trials,  if  necessary.  Successive  iterations  of 
the  model  are  run  to  calibrate  the  model's  algorithms  and  model  data.  The 
modeler  must  be  careful  not  to  calibrate  the  model  to  a  specific  trial,  as 
the  goal  is  to  have  the  model  represent  the  field  test  within  specified 


statistical  tolerances.  These  tolerances  are  usually  defined  by  the  agency 

responsible  for  accrediting  the  simulation  model. 
5.    Phase  IV  (Validation/Accreditation) 

In  phase  four,  the  "calibrated"  model  is  used  to  extrapolate  the 

test  results  to  conditions,  scenarios  and  threats  that  were  not  tested  due 

to  cost,  time,  or  equipment /environmental  constraints.  This  occurs  after 

the  model  is  validated  and  accredited.   Validation  of  a  model  is  the 

process  of  determining  that  a  model  is  an  accurate  representation  of  the 

intended  real -world  entity  from  the  perspective  of  the  intended  use  of  the 

model  [ARMY  92] .  Accreditation  is  performed  by  OPTEC,  based  on  experience 

and  expert  judgement  that  the  model  is  adequate  for  its  intended  use  [ARMY 

92].   It  is  the  certification  that  the  model  is  acceptable  for  use  for  a 

specific  type  of  application  [ARMY  92].   The  credibility  of  the  extended 

results  naturally  depends  on  how  far  an  extension  is  from  a  calibration 

point  (e.g.,  scenario  in  jungle  terrain  extended  to  desert  terrain  or 

Mission  Oriented  Personnel  Posture  (MOPP)   0  to  MOPP  4).    Thus,   the 

modeler/analyst  must  take  care  not  to  extend  the  model  results  to  a  point 

where  the  tactical  character  of  the  battle  changes  so  much  that  the 

calibration  representation  is  no  longer  valid. 

This  thesis  will  support  TRAC-MTRY's  ongoing  accreditation  effort  for 

M-T-M  using  the  Janus (A)  high  resolution  combat  simulation  model.   The 

focus  will  be  to  compare  and  analyze  the  effects  of  the  Janus (A)  terrain 

and  the  actual  terrain  characteristics  on  the  modeled  operational  test. 

The  detection  range  of  an  enemy  vehicle  by  the  opposing  force  will  serve 

as  the  MOE. 


III.   TERRAIN  SIMULATION  PROCESS 

A.    JANUS (A)  COMBAT  MODEL 
1 .    Overview 

Janus,  named  for  the  two-faced  Roman  god  who  guards  portals,  is 

an  interactive,  computer-based,  war-gaming  simulation  for  combat  operations 

at  the  brigade  and  lower  level  (JANU  93].   The  Janus  combat  simulation 

model  has  evolved  through  several  versions,  beginning  with  the  Janus (L) 

version  that  was  developed  by  the  Lawrence  Livermore  National  Laboratory 

(LLNL) .  The  U.S.  Army  TRADOC  Analysis  Command,  White  Sands  Missile  Range, 

New  Mexico  (TRAC-WSMR)  acquired  this  prototype  and  developed  Janus (T)  to 

meet  combat  development  needs.   From  the  Janus (T)  version  came  the  Janus 

Army  or  Janus (A)  version  which  is  used  for  combat  development  and  training 

needs.   Janus (A) ,  version  3.0,  is  the  most  current  version  now  in  use  by 

the  Army.   Today,  Janus (A)  is  developed,  maintained  and  distributed  by 

TRAC-WSMR,  and  is  fielded  throughout  the  Army  as  a  tool  for  trainers  and 

analysts  in  testing,  research  and  combat  development. 

Janus (A)  is  a  "high  resolution,  event-driven,  two-sided,  closed, 

stochastic,  ground  combat  simulation"  [JANU  93]. 

•  'High  resolution'  means  that  the  smallest  object  modeled  in  the 
simulation  is  an  individual  weapon  system  (e.g.,  a  soldier  and  his  M16 
rifle) . 

•  'Event  driven'  means  the  state  of  the  simulation  is  updated 
asynchronously  when  a  combat  event  occurs. 

•  'Two-sided'  refers  to  the  two  opposing  forces,  Blue  and  Red.  The  Blue 
force  and  Red  force  are  simultaneously  directed  on  separate  monitors 
by  two  different  sets  of  players.  Janus (A)  can  be  used  in  a  single 
player  mode  as  well. 

•  'Closed'  means  the  disposition  of  the  opposing  force  is  unknown, 
except  those  locations  found  by  direct  intelligence  reports  from 
friendly  forces. 

•  'Stochastic'  means  that  certain  events  are  not  predetermined,  but 
occur  according  to  the  laws  of  probability. 


•  'Ground  combat"  means  the  principal  focus  of  the  simulation  is  on 
ground  maneuver  and  artillery  combat  units. 

Janus (A)  is  also  capable  of  simulating  the  effects  of  weather,  rotary  and 

fixed  wing  aircraft,  day  and  night  visibility,  chemical  environment, 

minefield  employment  and  other  variables.  Janus (A)  is  written  entirely  in 

the  VAX-11  FORTRAN  language  and  currently  runs  on  any  Digital  Equipment 

Corporation  (DEC)  VAX  family  computer  using  the  standard  VMS  operating 

system. 

2.    Janus (A)  Battlefield  Terrain 

The  Janus (A)  terrain  database  is  a  digitized  representation  of  a 

portion  of  real  world  terrain,  based  on  Digital  Terrain  Elevation  Data 

(DTED)  Level  I  data.   The  DTED  Level  I  data  is  supplied  by  the  Defense 

Mapping  Agency  (DMA)  .   The  data  is  converted  from  a  profile  plot  to  a 

contour  plot  for  use  in  the  Janus (A)  computer  simulation.  Figure  2. 
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Figure  2  Digital  Terrain  Elevation  Data  Level  I 

Janus (A)  provides  the  user  a  computer  generated  map  image  of  the  exercise 
area,  showing  terrain  contours,  forested  areas,  roads,  rivers  and  urban 
areas.  Within  Janus  (A)  ,  the  user  can  initialize  the  size  of  the  exercise 
area  based  on  his  needs.  The  exercise  area  is  a  square  that  is  subdivided 
into  360,000  (600  x  600)  grid  cells.  The  user  can  vary  the  size 
(resolution)  of  the  grid  cells  from  25m  x  25m,  50m  x  50m,  100m  x  100m  to 
200m  x  200m.  Each  grid  cell  carries  the  characteristics  of  the  terrain: 
elevation,  vegetation  or  urban  area,  density  and  height,  road  data,  rivers 
and  bridges  within  that  grid  cell.  Each  characteristic  is  assigned  a 
numeric  value  that  defines  the  characteristic  within  that  grid  cell.  For 
example,  a  numeric  value  of  five  for  vegetation  defines  the  density  and 
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height  of  the  vegetation  in  that  grid  cell  as  a  representation  of  the 
actual  tree  height  and  density  for  that  geographic  area.  All  of  these 
characteristics  are  assumed  constant  throughout  the  grid  cell.  By  reducing 
the  size  of  the  grid  cells,  the  user  is  able  to  better  represent  changes 
in  the  elevation,  vegetation,  urban  areas,  road  networks  and  rivers  which 
should  better  simulate  the  real  world  terrain  being  modeled.  This  thesis 
will  explore  this  question  in  Chapter  IV  by  running  the  Javelin  scenarios 
on  the  generic  Janus (A)  terrain  and  on  the  modified  50  meter  resolution 
"real  world"  terrain  (described  below) . 

B .    PEGASUS 

1 .    Overview 

The  Perspective  View  Generator  and  Analysis  Systems  for  Unmanned 

Sensors  (PEGASUS)  is  a  prototype  terrain  database  creation  system.   It  is 

being  used  by  the  Army  Test  and  Experimentation  Command   (TEXCOM)  , 

Experimentation  Center  (TEC)  at  Fort  Hunter  Liggett,  California  to  produce 

object  material  descriptors  for  the  PEGASUS  real-time  perspective  view 

generation  replicator  [BAER  91].   The  object  material  descriptors  are  the 

information  such  as  height,  width,  tree  canopy  shape  and  other  attributes 

that  describe  various  objects  such  as  trees,  buildings  and  vehicles.   "The 

replicator  is  to  provide  substitute  video  images  that  are  realistic 

representations  of  the  battle  maneuvers  actually  occurring  in  real  time" 

[BAER  91]  .  The  replicator  can  be  perceived  as  a  data  flow  through  a  sensor 

response  system. 

The  system  flow  starts  with  photographic  measurements  of  targets, 
cultural  features  and  terrain  background.  To  these  are  added 
geographic  and  object  size  measurements  to  orient  and  scale  the 
photographic  data.  The  photographic  input  data  is  digitized, 
radiometrically  calibrated,  registered  with  location  and  scale 
information  and  stored  on  optical  disk  in  a  registered  image 
database  [BAER  91]. 
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2.    Terrain  Database  Creation 

The  PEGASUS  database  creation  system  is  a  transputer-based 
computer  system,  with  numerous  algorithms  designed  to  input,  measure  and 
parameterize  visual  images  into  an  object-oriented  database.  The 
transputers  provide  the  parallel  processing  necessary  to  create  the  visual 
imagery  of  this  complex  database  in  close  to  'real-time".  The  terrain 
database  creation  software  is  depicted  in  Figure  3. 
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Figure  3  Terrain  Database  Creation  System  Diagram 

The  diagram  shows  the  variety  and  flow  of  information  used  to  create  and 
update  a  PVG  database.  The  system  can  use  the  elevation  and  vegetation 
code  data  supplied  by  DMA  and  the  Waterways  Experimentation  Station  (WES) 
respectively,  or  generate  its  own  initial  estimates.  The  result  is  an  one 
meter  resolution  terrain  database  for  a  375  square  kilometer  test  area. 
This  one  meter  terrain  database  can  be  expanded  to  a  four  meter,  sixteen 
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meter  and  sixty  four  meter  database  by  filtering  and  averaging  high 
resolution  information. 

C.    SCENARIO  TERRAIN  FILE 

The  scenario  terrain  file  is  the  computer  simulated  terrain  that 

represents  the  actual  test  site  terrain.  Within  Janus (A)  exist  the  Master 

Terrain  Files  (MTF)  which  are  based  on  DMA  DTED  files.   The  analyst  or 

trainer  can  customize  his  specific  scenario  terrain  file  using  the  Janus (A) 

terrain  editor  which  is  part  of  the  Janus (A)  Database  Manager.    This 

customizing  process  determines  the  modeled  test  site  terrain,  and  thus 

determines  the  Lines  of  Sight  (LOS)  and  movement  factors.   Each  scenario 

terrain  file  consists  of  a  terrain  file  (TERAIN### .DAT; # )  and  a  terrain 

screen  file  (TSCRN### .DAT; # ) .     This  section  will  explain  the  processes 

used  to  make  the  Janus (A)  and  PEGASUS  simulated  terrain  and  discuss  the 

role  and  products  of  agencies  involved  in  the  creation  of  model  terrain. 
1.    Terrain  File  (TERAIN##.DAT;#) 

The  terrain  file  contains  the  digitized  data  derived  from  data 

supplied  by  DMA. 

a.    DMA   Process 

As  discussed  earlier,  The  data  supplied  by  DMA  is  called  the 

Level  I  Digital  Terrain  Elevation  Data  (DTED)  which  is  used  for  all 

military  activities  and  systems  that  require  landform,  slope  and  elevation 

in  a  digital  format.   The  DTED  Level  I  data  is  derived  from  imagery  with 

a  one  degree  by  one  degree  cell  size  defined  by  the  integer  one  degree 

latitudes  and  one  degree  longitudes  of  the  geographic  reference  system 

(DIGI  90]  .  The  vertical  and  horizontal  datums  are  Mean  Sea  Level  (MSL)  and 

World  Geodetic  System  (WGS)  84,  respectively.    The  elevation  data  is 

expressed  in  meters  and  the  information  content  is  equivalent  to  a 

1:250,000  scale  resolution.   DMA  provides  this  data  in  a  matrix  structure 

with  an  ASCII  labeled  variable  length  record  format. 
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Jb.    PEGASUS   Process 

PEGASUS  can  use  the  DTED  Level  I  data  supplied  by  DMA  as  a 
baseline  for  the  terrain  file  or  generate  its  own.  Using  the  DTED  Level 
I  data  as  a  baseline  creates  no  additional  error  problems  because  of  the 
terrain  refinement  process  in  PEGASUS.  Within  PEGASUS,  the  Perspective 
View  Database  (PVDB)  details  the  test  scenario  terrain  area  in  resolutions 
of  one,  four,  sixteen  and  sixty  four  meters  and  covers  a  32.768  x  28.672 
kilometer  rectangular  area.  The  coordinates  of  the  rectangle  conform  to 
the  Universal  Transverse  Mercator  (UTM)  conventions.  The  PVDB  is  organized 
into  a  Tile/Block/Post  (TBP)  structure.  The  PVDB  is  broken  up  into  an  8 
x  7  collection  of  tiles,  Figure  4,  and  each  tile  is  partitioned  into  a  16 
x  16  array  of  blocks  and  each  block  contains  an  arrangement  of  data  posts, 
Figure  5  [AKIN  89].  The  posts  contains  the  elevations,  cultural  feature 
indicators,  gray  shade  values,  surface  normal  indicators  and  sun  shade 
flags  required  for  the  creation  of  the  detailed  terrain  information. 


15 


7  TILES 


o> 


O 
O 


X 

2  18  18  (2  i5 

s 

sis 

e    i  © 
*    1  > 

S  1  8  !  8 

e   1  e 

1 

«     a* 

1     © 

-  !  -  1  -  i 

-      i     N     I     U     , 

"■!    I    ! 

1  S  is 

Is 

a  !»  ! 

i    i   i           s 

»|* 

is  is  i 

I 
i 

! 

*  i  % 

ill     !    ! 

! 

1        '        i        i 

i 

i 

is!     ! 

!    :    1    :    ■    ■    !    j    ! 

■  ii 

it!    j    ;    ! 

i    i       : 

•  1 

:*!     i          i 

i  ©  i 

j 

III! 

; 

irJ    ;   i 

i 
1 

1     !     ! 

; 

u 

It 

!    | 

ill!: 

; 

j  * 

1    n 

1*1  i  i  I 

1 

i  ■  '  1 

i 

1  $ 

is!    I 

!    !    :    1    1    I 

1 « 

i  o  !  2  i      1 

!  A  !        i 

1 

1 

i        :    1    !    i 

ig|2ISl 

!    i    i      ...    II 

Sj  5 

;  S  1  2  1  B  !  8 

!    ! 

1                 t                 I   m  |    m 
i         !        !        1         |   O  1  O 

tn  1  m 
m  1  n 

i©.-i»»i«i*|w;« 

-  !  • 

H    1    n 

•  :  > 

- 

S3 

16  BLOCKS 


> 
CO 

> 

CO 

m 


m 


Figure  4  PVDB  Tile  Structure 
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Figure  5  PVDB  Block  Structure 

A  tile  represents  an  area  of  ground  measuring  4096  x  4096  meters  and  a 
block  represents  an  area  measuring  256  x  256  meters.  The  following  table 
shows  the  number  of  data  posts  in  a  block,  which  is  dependent  on  the  data 
resolution: 

Table  I   POSTS  WITHIN  A  BLOCK 


Resolution 

#    of    Posts 

64   meter 

4   x        4 

16  meter 

16   x      16 

4   meter 

64    x      64 

1   meter 

256   x   256 
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2.    Screen  File  (TSCRN###.DAT;ft) 

This  file  contains  data  depicting  vegetation,  grid  squares, 

roads,  rivers,  contour  lines  and  urban  areas. 

a.  Waterways  Experimentation  Station    (WES)    Process 

The  Waterways  Experimentation  Station  is  part  of  the  Corps 
of  Engineers  and  has  a  mission  of  supporting  the  U.  S.  Army  by  means  of 
geotechnical  support.  Terrain  database  development  began  with  the  creation 
of  descriptions  of  basic  terrain  factors  that  effect  the  movement  of 
mounted  or  dismounted  units  engaged  in  combat.  WES ' s  procedure  for 
developing  terrain  databases  involves  preparing  overlays  that  describe 
basic  terrain  factors  of  the  specified  test  area  [BULL  88]  .  These  overlays 
are  called  primary  factor  maps.  The  primary  factors  associated  with  each 
basic  terrain  factor  are:  land  use,  slope,  soils,  obstacles  and  linear 
features.  A  listing  of  the  components  of  the  primary  factors  can  be  found 
in  Appendix  A.  The  primary  factor  maps  are  digitized  and  rasterized  at  the 
required  resolution  and  combined  with  the  terrain  file  to  form  the  scenario 

terrain  database. 

b.  PEGASVS  Process 

Only  one  process  is  used  to  produce  the  terrain  file  and 
screen  file  in  PEGASUS.  The  PVDB  contains  the  cultural  feature  indicators 
as  well  as  the  elevations.  This  is  accomplished  through  the  use  of  a  32- 
bit  terrain  element  database  that  provides  the  information  for  surface 
objects  such  as  rocks,  trees,  buildings,  etc.  The  terrain  database 
creation  diagram,  Figure  3,  again  shows  the  process  for  building  and 
continuously  updating  the  scenario  terrain  database. 

D.    TERRAIN  MODIFICATION 

All  direct  fire  weapons  are  dependent  on  lines  of  sight  (LOS)  to  be 
effective,  and  Javelin  is  a  direct  fire  weapon.  This  simple,  but  crucial 
fact  makes  the  representation  of  the  actual  terrain  in  the  simulation  model 
very  important.  The  characteristics  of  the  terrain  are  a  key  factor  in 
LOS  determination,  and  the  LOS  is  a  key  factor  in  the  detection  of  any 
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enemy  element.  There  are  two  factors  that  effect  LOS  on  the  ground:  [CELS 
92] 

•  Elevation:  The  elevation  level  between  forces  must  allow  for  LOS.  If 
intervening  crests  mask  the  LOS  between  the  sensor  and  the  target,  no 
direct  fire  occurs. 

•  Vegetation  and  urban  areas:  Trees  and  man-made  objects  can  interfere 
with  LOS.  So,  if  vegetation  and  urban  areas  exist,  they  must  be 
accurately  represented  in  the  model . 

Janus (A)  addresses  both  deterministic  and  probabilistic  aspects  of 
detecting  a  target.  It  addresses  the  deterministic  aspect  by  calculating 
whether  there  is  any  type  of  terrain  mask  between  the  sensor  and  the 
target.  The  probabilistic  aspects  are  addressed  by  determining  if  the 
target  can  be  detected  through  vegetation  or  urban  areas,  by  considering 
the  density  of  the  objects  in  the  line  of  sight. 

It  has  already  been  explained  that  the  density  and  height  of  an  object 
are  assumed  constant  and  uniform  throughout  a  grid  cell.  However,  the 
program  modelers  of  Janus (A)  also  modeled  an  area  or  group  of  grid  cells, 
that  has  a  stand  of  trees  as  an  area  having  homogeneous  density  and  tree 
heights.  A  color  picture  of  this  Janus (A)  representation  for  the  Javelin 
test  area  is  found  in  Appendix  B.  So,  to  ensure  that  the  LOS  for  the 
Javelin  correctly  reflected  the  actual  terrain,  this  study  modified  the 
Janus (A)  scenario  terrain  using  an  eight  meter  terrain  database  derived 
from  the  PEGASUS  one  meter  terrain  database.  The  eight  meter  terrain 
database  depicts  the  Fort  Hunter  Liggett  area  and  was  adapted  to  run  on  a 
personal  computer.  Within  this  database,  the  user  is  able  to  zoom  in  from 
a  top  down  perspective  to  see  individual  trees.  Also,  when  the  user  clicks 
the  mouse  on  a  particular  piece  of  terrain,  the  military  grid  coordinates 
along  with  an  elevation  are  displayed.  This  allows  the  user  to  more 
accurately  place  vegetation  on  the  Janus (A)  simulated  terrain.  The 
database  is  now  the  possession  of  TRAC-MTRY  and  the  Janus (A)  developers  at 
TRAC-WSMR  are  attempting  to  modify  the  database  program  files  so  the 
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Janus (A)  Terrain  Research  Program  (TRP)  can  read  the  files  into  a  Janus (A) 
two-dimensional  display  of  the  Fort  Hunter  Liggett  area. 

Next,  an  eight  kilometer  square  area  covering  the  Javelin  test  site 
area  was  modeled  in  Janus (A) .  Two  copies  of  this  scenario  terrain  were 
stored  in  separate  files.  Both  files  have  a  resolution  of  fifty  meters. 
Then  one  of  the  files  was  modified  using  the  eight  meter  PEGASUS  database. 
In  this  scenario  file,  a  vegetation  code  was  placed  on,  or  removed  from, 
each  fifty  meter  grid  cell  and  the  density  of  each  grid  cell  was  also 
changed,  if  needed.  Subjective  judgement  was  involved  in  the  areas  of 
density  and  tree  height  and  whether  or  not  a  grid  cell  needed  trees  in  it. 
The  tree  heights  can  be  changed  within  the  terrain  editor.  However,  there 
is  a  basic  problem  with  the  way  the  tree  heights  are  handled.  The  tree 
heights  are  tied  to  the  density  codes.  This  means  that  for  a  density  code 
of  five  a  corresponding  tree  height  can  be  set  to  eleven  meters  or  any 
other  value  you  pick.  The  problem  is,  the  tree  height  then  applies  for  all 
densities  of  five  in  the  scenario  terrain.  The  model  does  not  take  in 
consideration  that  different  types  of  trees  can  be  found  in  the  same 
scenario  area  (e.g.,  deciduous  stand  of  trees  and  an  orange  grove)  or  a 
younger  stand  of  trees  may  populate  a  certain  sector  of  the  scenario 
terrain.  For  the  test  site  area,  the  majority  of  the  trees  observed  on  two 
different  terrain  walks  were  approximately  nine  to  thirteen  meters  high. 
This  was  verified  by  forestry  sources  on  the  post.  There  was  a  problem  of 
determining  whether  trees  were  needed  in  a  certain  grid  cell  and  the 
correct  density  of  trees  for  a  particular  grid  cell.  This  was  solved  be 
looking  at  the  number  of  trees  per  fifty  meter  square  area  and  how  close 
the  trees  were  to  each  other. 

Although,  subjective  judgement  was  used  to  modify  Janus (A)  terrain, 
the  belief  is  that  the  modified  terrain  will  better  represent  the  actual 
test  site  terrain.  A  color  picture  of  the  default  and  modified  terrain  can 
be  found  in  Appendix  B. 
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E.    JANUS (A)  SUFFICIENCY  TO  MODEL  REAL  WORLD  TERRAIN 

As  stated,  Janus  (A)  is  widely  used  throughout  the  Army  to  model 
brigade  and  lower  combat  operations.  Even  though  Janus (A)  is  an  approved 
simulation  model  for  the  above  use,  is  Janus (A)  sufficient  to  model  an 
operational  test?  This  thesis  assumes  that  the  Janus (A)  model  terrain  can 
sufficiently  represent  real  world  terrain  for  modeling  purposes. 

Currently,  there  are  two  methods  for  modeling  terrain.  Both  of  these 
methods  have  been  proven  to  represent  real  world  terrain  adequately.  The 
most  widely  used  method  is  the  plateau  method.  Janus (A)  uses  the  plateau 
method.  In  this  method  grid  cells  encapsulate  the  terrain  characteristics, 
and  one  elevation  value  is  assigned  to  each  cell.  At  the  respective 
resolution  (e.g.,  50x50  meter)  a  system  of  pilings  with  flat  tops 
(plateaus)  are  created  which  represent  the  real  world  terrain.  This  causes 
a  stepwise  jump  or  drop  transition  to  each  grid  cell.  Real  world  terrain 
flows  in  a  more  continuous  fashion  except  at  cliffs  and  sinkholes. 
However,  as  the  resolution  is  increased  to  twelve  meters  or  six  meters  the 
plateau  method  smooths  out  considerably,  depending  on  the  area  being 
modeled.  This  method  also  lends  itself  to  translation  into  UTM  grid 
coordinates  which  are  the  bases  for  the  military  grid  coordinate  system. 

The  other  method  of  modeling  terrain  is  called  the  polygonal  method. 
In  this  method,  straight  lines  are  connected  to  designated  elevation 
points.  This  creates  a  series  of  slopes  that  can  be  translated  into  a 
digital  contour  map.  The  contour  map  is  then  overlayed  with  a  digitized 
screen  file  with  the  result  that  represents  real  world  terrain.  However, 
this  method  does  not  lend  itself  to  easy  translation  into  the  military  grid 
coordinate  system. 
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XV.   ANALYSIS  AND  DISCUSSION  OP  JANUS (A)  MODEL  TERRAIN 

A .    GENERAL 

The  analysis  was  conducted  in  the  last  two  phases.  In  phase  two,  the 
vegetation  code  distributions  in  the  default  and  modified  terrain  databases 
were  compared  to  see  if  the  differences  were  more  substantial  than  would 
be  expected  because  of  natural  variation.  In  the  third  phase,  three  actual 
IOTE  scenarios  were  simulated  using  each  database  to  see  if  the  test 
results  would  be  effected.  This  phase  was  done  because  the  vegetation 
codes  for  the  terrain  files  are  different. 

For  the  analysis,  a  sample  one  kilometer  square  was  extracted  from 
each  terrain  file.  These  terrain  squares  represented  the  same  patch  of 
land  in  each  file.  Using  the  Janus (A)  terrain  utilities,  the  vegetation 
codes  for  each  one  kilometer  sample  were  recorded  for  analysis.  In 
Janus  (A)  the  vegetation  codes  can  only  take  the  the  value  of  discrete 
integers  from  zero  to  seven.  A  total  of  800  vegetation  data  points  were 
collected,  400  for  each  terrain  sample.  These  observations  represent  the 
entire  vegetation  code  population  for  each  selected  one  kilometer;  the  same 
one  kilometer  square  was  chosen  from  each  database.  The  question  to  be 
answered  was  not  whether  the  two  populations  were  different,  the  known 
changes  ensured  that  they  were  different.  But,  did  the  changes  that  were 
required  to  make  the  modified  one  kilometer  square  terrain  more  accurately 
reflect  the  actual  terrain  result  in  differences  more  substantial  than 
would  be  expected  between  two  one  kilometer  squares  chosen  at  random. 
Parametric,  nonparametric  and  graphical  statistical  analysis  techniques 
were  used  to  compare  and  contrast  the  vegetation  code  distributions.  The 
analytical  software  packages  used  in  this  thesis  were  the  statistical 
package  SPSS  and  A  Graphical  Statistical  System  (AGSS) .  AGSS  is  available 
for  use  at  the  Naval  Postgraduate  School  (NPS)  under  a  test  site  agreement 
with  IBM  Research.  We  are  indebted  to  Dr.  Peter  Welch  for  making  this  possible. 
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First,  graghical  tests  for  normality  were  conducted  on  the  vegetation 
code  data  from  each  sample  to  determine  if  parametric  techniques  were 
appropriate.  As  can  be  seen  in  the  following  section,  neither  vegetation 
code  distribution  is  well  approximated  by  the  normal  distribution. 
Therefore,  nonparametric  and  graphical  tools  were  used  in  the  analysis. 
Note  that  it  is  acceptable  to  use  nonparametric  techniques  for  data  that 
does  follow  the  normal  distribution.  Nonparametric  procedures  perform 
almost  as  well  as  the  t  and  F  tests  for  normal  data  ,  and  will  often 
perform  much  better  under  non-normal  conditions  [DEVO  87] .  Graphical  tools 
were  used  to  provide  visual  insights  into  the  data  distributions. 

Next,  three  scenarios  from  the  Initial  Operational  Test  and  Evaluation 
(IOTE)  for  the  Javelin  antitank  weapon  were  run  on  each  the  default  and 
modified  terrain  files.  The  scenarios:  a  night  deliberate  defense,  a  day 
hasty  defense  and  a  day  deliberate  defense  were  set  up  to  be  run  on  the 
Janus (A)  simulation  model  by  jCPT  Mick  McGuire  for  his  thesis  The  Javelin 

Versus the Draoon  II.   A  Comparative  Analysis.    These  scenarios  were 

imported  from  CPT  McGuire 's  database  in  a  cooperative  effort  to  improve 
the  Javelin  IOTE.  The  three  scenarios  were  each  run  eight  times  on  each 
terrain  file  for  a  total  of  48  runs.  The  measure  of  effectiveness  (MOE) 
was  the  detection  range  of  an  enemy  target  by  either  force.  This  MOE  was 
used  because  detection  is  greatly  effected  by  the  vegetation  represented 
in  the  terrain  database. 

B.    ANALYSIS  OP  TERRAIN  VEGETATION 

The  first  test  on  the  terrain  samples  was  a  test  to  determine  if  the 
terrain  sample  vegetation  codes  were  from  the  normal  distribution.  Normal 
probability  plots  and  detrended  normal  plots  were  was  used  to  determine  if 
nonparametric  or  parametric  statistical  procedures  should  be  used  to 
analyze  the  data.  The  normal  probability  plot  pairs  each  observed  value 
with  its  expected  value  from  the  normal  distribution.  If  the  sample  is 
from  the  normal  distribution  the  points  should  fall  more  or  less  on  a 
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straight  line  [MEND  90] .  Figure  6  shows  that  the  points  do  not  cluster 
around  a  straight  line  and  in  Figure  7  shows  that  the  deviations  from  the 
straight  line,  the  detrended  residuals,  are  not  randomly  distributed  around 
zero. 


Normal  Plot  of  VEGETATION  CODE 

for  JANUS (A)  DEFAULT  TERRAIN 
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Figure   6     Default  Normal   Plot 


Detrended  Normal  Plot  of  VEGETATION  CODE 

FOR  JANUS (A)  DEFAULT  TERRAIN 
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Figure  7   Default  Detrended  Normal  Plot 

Likewise,  the  data  points  for  the  Janus (A)  modified  terrain  do  not 
fall  along  the  straight  line  in  Figure  8,  and  the  detrended  normal  plot, 
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Figure  9,  shows  a  lack  of  randomness  in  the  dispersion  of  the  residuals 


Normal  Plot  of  VEGETATION  CODE 
for  JANUS (A)   MODIFIED  TERRAIN 
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Figure   8     Modified  Normal   Plot 


Detrended  Normal  Plot  of  VEGETATION  CODE 

for  JANUS (A)  MODIFIED  TERRAIN 
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Figure  9  Modified  Detrended  Normal  Plot 

These  four  plots  indicate  that  the  vegetation  code  data  in  the  default 
and  modified  Janus (A)  terrain  do  not  follow  the  normal  distribution. 
Therefore,  nonparametric  and  graphical  techniques  will  be  used  as  the 
analysis  tools.  By  looking  at  the  graphs  it  cannot  easily  be  determined 
that  400  data  points  per  terrain  sample  were  used  in  this  analysis  process. 
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This  is  because  the  vegetation  codes  can  only  take  the  value  of  discrete 
integers  from  zero  to  seven.  Therefore,  the  data  for  both  terrain  samples 
was  plotted  using  a  bar  graph.  Figure  10  graphically  confirms  that  the 
vegetation  codes  do  not  follow  the  normal  distribution,  and  further 
indicates  that  their  distributions  are  not  similar. 


COMPARISON  OF  DEFAULT  AND  MODIFIED 
VEGETATION  CODES 
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Figure  10  Comparison  of  Vegetation  Codes 

In  sum,  the  preceding  five  graphs  indicated  that  the  vegetation  codes 
for  the  default  and  modified  terrain  did  not  follow  the  normal 
distribution.  Therefore,  the  assumption  of  normality  could  not  be  made  and 
nonparametric  and  graphical  techniques  were  used  to  analyze  whether  the 
vegetation  codes  in  the  two  terrains  files  were  similar.  The  Mann-Whitney 
test  was  used  to  test  the  null  hypothesis  that  the  population  relative 
frequency  distributions  for  the  two  terrain  samples  are  identical  [MEND 
90]  .  The  least  restrictive  alternative  is  that  one  distribution  is 
stochastically  larger  than  the  other  [GIBB  92].  The  Mann-Whitney  test 
requires  the  assumption  of  two  independent  random  samples  from  continuous 
distributions.  The  vegetation  code  samples  clearly  do  not  satisfy  that 
assumption.  However,  the  test  results  do  answer  the  question;  "Would  two 
independent  random  samples  from  the  same  vegetation  code  distribution  be 
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expected  to  exhibit  differences  similar  to  those  between  the  default  and 
modified  terrain  samples?"  The  answer  is  clearly  no.  The  following  table 
shows  the  results  of  the  Mann-Whitney  test. 


Table  II   VEGETATION  CODE 

MANN-WHITNEY  TEST 

Mann-Whitney  U  Test 

VEGCODE 

by  TERRAIN 

Mean  Rank 

Cases 

376.41 
424.59 

400 
400 

TERRAIN  =  Modified 
TERRAIN  =  Default 

800 

Total 

U 
70365. 

W 
0       150565.0 

Corrected  for  ties 

Z      2-Tailed  P 
-3.1360        .0017 

After  the  vegetation  codes  are  merged  and  ranked,  the  U  statistic 
represents  the  number  of  times  a  modified  terrain  vegetation  code  value 
preceeds  a  value  for  the  default  terrain  vegetation  code.  The  very  large 
value  of  U  indicates  there  is  a  separation  of  the  ordered  modified  and 
default  vegetation  code  observations  and  indicates  a  stochastic  difference 
between  the  distributions.  Since  the  significance  level  is  small  (less 
than  .025,  for  a  two  tailed  test)  the  null  hypothesis  that  the  two 
vegetation  code  distributions  are  the  same  is  rejected.  It  is  concluded 
that  the  vegetation  codes  in  the  Janus (A)  modified  terrain  are 
stochastically  different  from  the  vegetation  codes  in  the  Janus (A)  default 
terrain. 

The  distrbutions  of  the  two  vegetation  code  samples  are  not  known,  but 
the  following  graph.  Figure  11,  provides  some  detailed  comparisons  of  the 
distributions  of  the  two  data  sets.  The  graph  is  an  empirical  quant ile- 
quantile  plot  and  is  constructed  by  plotting  the  quantiles  of  one  empirical 
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distribution  against  the  corresponding  quantiles  of  the  other  [CHAM  83]. 
If  the  distributions  were  identical,  all  the  points  would  lie  on  the  line 
y  =  x.  The  points  do  not  lie  on  the  y  =  x  line,  thus  the  distributions  are 
not  identical.  Because  most  of  the  points  lie  below  the  y  =  x  line,  it  is 
clear  that  the  Janus (A)  default  terrain  has  the  higher  vegetation  density 
codes.  Also,  because  the  data  points  are  discrete  integers  they  form  a 
stepwise  linear  curve. 


O-O  PLOT  COMPARISON  OF  VEGETATION  CODES  FOR  THE 
JANUS(A)  DEFAULT  AND  MODIFIED  TERRAIN 
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Figure  11     Vegetation   Code 
Quantile-Quantile  Plot 


C.    ANALYSIS  OP  THE  SCENARIO  RUNS 

It  has  been  determined  that  the  vegetation  codes  in  the  two  terrain 
files  are  stochastically  different.  Intuition  indicates  that  if  the  same 
scenario  was  run  on  each  terrain  file,  the  detection  ranges  should  also 
come  from  different  distributions.  This  section  will  explore  this 
hypothesis.  In  this  analysis,  vegetation  code  distribution  will  be  treated 
as  a  factor  at  two  levels:  default  and  modified. 

The  scenarios  consisted  of  a  U.S.  light  infantry  company  team  -  BLUE 
force  and  a  North  Korean  mechanized  platoon  -  RED  force.  The  light 
infantry  team  is  equipped  with  M16  rifles,  a  60mm  mortar  section,  squad 
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automatic  weapons  (SAW),  M60  machine  guns,  light  antitank  weapons  (LAW)  and 

Javelin  medium  antitank  weapons.   The  company  team  is  also  augmented  with 

a  heavy  mortar  section.   The  North  Korean  platoon  is  organized  with  three 

T72  tanks,  three  BMPs  and  a  heavy  mortar  section.   The  BLUE  and  RED  forces 

have  119  and  41  personnel,  respectively.   The  tactical  employment  of  the 

forces  follows  the  known  warfighting  doctrine  of  each  country. 

To  prevent  a  learning  effect  when  running  the  scenarios,  each  run  was 

conducted  using  the  Automatic  Janus (A)  mode.   In  this  mode,  the  computer 

runs  the  scenario  on  saved  puck  movements  (player  actions)  which  were  made 

by  the  initial  man  in  the  loop  interactive  battle.   The  probabilistic 

outcomes  (detection,  kill,  etc.)  are  not  necessarily  the  same,  so  the  data 

can  be  viewed  as  independent,  random  samples. 
1.    The  Night  Deliberate  Defense 

The  first  scenario  was  the  night  deliberate  defense.   In  the 

terrain  area  where  this  battle  took  place,  there  were  many  modifications 

to  the  vegetation.  These  differences  were  captured  in  the  following 

descriptive  summary. 


Table  III    NIGHT  DELIBERATE  DEFENSE 
DETECTION  RANGE  SUMMARY 


DEFAULT 

MODIFIED 

MEAN 

2.52 

2.29 

S.E.  MEAN 

.03 

.04 

STD  DEV 

.92 

1.12 

MINIMUM 

.01 

.01 

MAXIMUM 

4.32 

4.46 

TOT  OBS 

715 

952 

Next  the  Mann  -  Whitney  test  was  used  to  test  the  null  hypothesis 
that  the  population  relative  frequency  distributions  for  the  two  detection 
range  samples  are  identical.  The  significance  level  was  .0004;  therefore, 
the  null  hypothesis  is  rejected.  The  population  relative  frequency 
distributions  for  the  detection  ranges  in  the  night  deliberate  defense  for 
the  two  terrain  files  were  significantly  different. 
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Table  IV   NIGHT  DELIBERATE  DEFENSE 

MANN -WHITNEY  TEST 

Mann-Whitney  U  Test 

DETECTRG 
by  TERRAIN 

Mean  Rank 

Cases 

882.43 
797.62 

715   TERRAIN 
952   TERRAIN 

1667   Total 

=  Default 
=  Modified 

U 
305710.5 

W 
630939.5 

Corrected  for  ties 
Z      2-Tailed  P 
-3.5601        .0004 

A  graphical  depiction  showing  the  differences  between  the  two 
detection  range  samples  is  shown  in  Figures  12  and  13  on  the  following 
page. 
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Detection  Range  for  Night  Deliberate  Defense 
(Frequency  vs.  Detection  Range) 
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Figure  12  Night  Deliberate  Defense  Detection  Ranges 
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Figure  13  Night  Deliberate  Defense  Detection  Ranges 

The  quant ile-quantile  plot  for  the  night  deliberate  defense. 
Figure  14,  shows  that  the  distributions  of  the  sample  detection  ranges  for 
the  default  terrain  and  the  modified  terrain  are  not  the  same.  Below  2.75 
kilometers  the  default  terrain  detection  ranges  were  greater. 
Interestingly,  the  detection  ranges  for  the  modified  terrain  are  similar 
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to  the  detection  ranges  for  the  default  terrain  for  distances  beyond  2.75 
kilometers.  This  could  be  explained  by  a  similarity  in  the  terrain 
vegetation  codes  for  the  particular  areas  modeled  and  the  use  of  infared 
sensors  to  detect  targets  at  these  ranges. 


0-0  PLOT  COMPARISON  OF  DETECTION  RANGES  FOR  THE 
NiCHT  DEUBERATE  DEFENSE 
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Figure  14     Detection   Range 
Quantile-Quantile  Plot 


2.    The  Day  Hasty  Defense 

The  second  scenario  was  the  day  hasty  defense.  The  terrain  in 
this  area  also  required  numerous  modifications  to  the  vegetation.  These 
differences  were  captured  in  the  following  descriptive  summary. 


Table  V   DAY  HASTY  DEFENSE  DETECTION 
RANGE  SUMMARY 


DEFAULT 

MODIFIED 

MEAN 

2.31 

1.37 

S.E.  MEAN 

.02 

.03 

STD  DEV 

.51 

1.06 

MINIMUM 

.03 

.01 

MAXIMUM 

2.89 

2.91 

TOT  OBS 

945 

1306 

Again  the  Mann  -  Whitney  test  was  used  to  test  the  null 
hypothesis  that  the  sample  population  relative  frequency  distributions  for 
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the  two  detection  range  samples  are  identical.  The  significance  level  was 
.0000.  The  population  relative  frequency  distributions  for  the  detection 
ranges  in  the  day  hasty  defense  for  the  two  terrain  files  were 
significantly  different. 


Table  VI   DAY  HASTY 

DEFENSE  MANN-WHITNEY  TEST 

Mann-Whitney  U  Test 

DETECTRG 
by  TERRAIN 

Mean  Rank 

Cases 

1396.58 
930.21 

945   TERRAIN 
1306   TERRAIN 

2251   Total 

=  Default 
=  Modified 

U 
361385.0 

W 
1319770.0 

Corrected  for  ties 
Z      2-Tailed  P 
-16.8017        .0000 

The  histograms  on  the  following  page  graphically  show  the 
differences  between  the  two  detection  range  samples.  Figures  15  and  16. 
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Detection  Range  for  Day  Hasty  Defense 
(Frequency  vs.  Detection  Range) 
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Figure  15     Day  Hasty  Defense  Detection  Ranges 
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300  t 


200  ' 


100  - 


0  -Oo      ,25        -5fl        -75        l-0o      l-2S      ^O      l-75     2-<>0     2-25      2 -50     2-7$ 


Std.  Dev  =  1.06 
Mean  =1.37 
N  =  1306.00 


DETECTRG 


Janus (A)  Modified  Terrain 


Figure  16  Day  Hasty  Defense  Detection  Ranges 

The  quantile-quantile  plot.  Figure  17,  for  the  day  hasty  defense 
shows  that  the  default  terrain  sample  detection  ranges  are  much  greater 
than  those  for  the  modified  terrain.  The  detection  range  distributions  for 
the  two  terrains  are  clearly  not  identical. 
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0-0  P10T  COMPARISON  Of  DETECTION  RANGES  TOR  THE 
DAY  HASTY  DEFENSE 


Figure  17     Detection 
Quantile-Quantile  Plot 


Range 


3.    The  Day  Deliberate  Defense 

In  the  last  scenario,  the  majority  of  the  engagement  area  was  in 
an  open  area.  Intuitively  the  modifications  to  the  terrain  should  have 
little  effect  on  the  detection  ranges  for  the  two  samples.  The  following 
descriptive  summary  indicates  the  similarities  between  the  two  detection 
range  samples. 


Table  VII     DAY   DELIBERATE   DEFENSE 
DETECTION  RANGE  SUMMARY 


DEFAULT 

MODIFIED 

MEAN 

1.69 

1.71 

S.E.  MEAN 

.04 

.03 

STD  DEV 

.95 

.86 

MINIMUM 

.08 

.02 

MAXIMUM 

3.81 

4.46 

TOT  OBS 

658 

983 

The  Mann  -  Whitney  test  showed  a  significance  level  of  .2267. 
The  null  hypothesis  that  the  population  relative  frequency  distributions 
for  the  two  detection  range  samples  are  identical  is  not  rejected. 
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Table  VIII   DAY  DELIBERATE  DEFENSE  MANN-WHITNEY  TEST 


Mann 

-Whitney  U  Test 

by 

DETECTRG 
TERRAIN 

Mean  Rank 

Cases 

838.28 
809.43 

658   TERRAIN 
983   TERRAIN 

1641   Total 

=  Default 
=  Modified 

U 
312034.0 

W 
551591.0 

Corrected  for  ties 
Z      2-Tailed  P 
-1.2089        .2267 

The  following  two  histograms  also  show  the  distinct  similarities 
between  the  detection  range  populations  for  the  two  samples,  Figures  18  and 
19. 
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Detection  Range  for  Day  Deliberate  Defense 
(Frequency  vs.  Detection  Range) 


100  ' 

♦"""■l"  1 

0, 

Std.   Dev  =  .9S 
Mean  =1.69 
N  x  658.00 


0.00  1.00  2.00  3.00  4.00 

.50  1.S0  2.50  3.50  4.50 


DETECTRG 


Janus(A)  Default  Terrain 


Figure  18     Day  Deliberate  Defense  Detection  Ranges 


Detection  Range  for  Day  Deliberate  Defense 
(Frequency    vs.    Detection  Range) 
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Figure  19  Day  Deliberate  Defense  Detection  Ranges 

In  the  day  deliberate  defense  scenario  the  distribution  of  the 
sample  detection  ranges  for  the  default  and  modified  terrain  are  not 
identical.  However,  they  are  similar  (Figure  20),  and  an  assumption  that 
the  distributions  are  equal  is  reasonable. 
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0-0  PLOT  COMPARISON  OF  DETECTION  RANGES  FOR  THE 
OAY  DELIBERATE  DEFENSE 


cctauu  tdoum  octcctkm  mmscs 


Figure  20     Detection   Range 
Quantile-Quantile  Plot 


4.   Aggregated  Detection  Ranges 

The  final  analysis,  was  conducted  on  the  aggregated  detection 
ranges  for  the  scenarios  on  the  default  and  modified  terrain  files.  After 
conducting  analysis  on  each  scenario  and  showing  that  they  are  different, 
it  seems  to  be  wrong  to  aggregate  the  data.  An  analysis  of  the  aggregated 
data  was  conducted  because  the  scenarios  of  an  IOTE  are  part  of  a  larger 
unit  operation.  These  operations  are  usually  72  or  96  hours  and  are 
composed  of  any  number  of  scenarios.  Thus,  the  aggregated  data  provides 
the  proponency  agency  insights  into  the  distribution  of  the  entire 
operational  scenario. 

The  results  of  the  analysis  showed  the  significance  level  to  be 
.0000.  The  population  relative  frequency  distrbutions  of  the  two  detection 
ranges  are  not  identical. 
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Table  IX   AGGREGATED  DETECTION  RANGE  MANN-WHITNEY  TEST 


Mann-Whitney  U  Test 

DETECTRG 
by  FILTER_$   Filter  Status 

Mean  Rank 

Cases 

3151.13 
2514.56 

2318   FILTER_$ 
3241   FILTER_$ 

5559   Total 

=  Default  Terrain 
=  Modified  Terrain 

Corrected  for  ties 

U 
2896033.0 

W 
7304326.0 

Z      2-Tailed  P 
-14.5814        .0000 

Figures  21  and  22  on  the  following  page  also  show  the  differences 
in  the  two  detection  samples. 
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Histogram  of  Detection  Range  for  Janus (A) 

Default  Terrain 
(Frequency  vs.  Detection  Range) 
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Figure  21  Default  Terrain  Detection  Ranges 


Histogram  of  Detection  Ranges  for  Janus (A) 

Modified  Terrain 

(Frequency    vs.  Detection  Range) 
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Figure  22  Modified  Terrain  Detection  Ranges 


The  quant ile-quantile  plot  of  the  aggregated  detection  ranges  for 
the  two  terrain  files,  Figure  23,  shows  that  their  distributions  are  not 
similar.  At  ranges  greater  than  than  2.8  kilometers  the  sample  detection 
range  distributions  are  not  identical  but  they  are  similar.  Below  2.8 
kilometers  the  detection  ranges  are  greater  for  the  default  terrain. 
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0-0  PLOT  COMPARISON  Of  THE  AGGREGATED  DETECTION 
RANGES  FOR  THE  DEFAULT  AND  MOOTED  TERRAIN 
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Figure  23     Detection   Range 
Quantile-Quantile  Plot 


D.    ANALYSIS  SUMMARY  OP  THE  SCENARIOS 

The  analysis  indicated  that  the  vegetation  code  distributions  for  the 
default  terrain  and  the  modified  terrain  are  different.  Also  the  detection 
range  distributions  for  the  two  terrain  files  in  each  scenario  were  not 
identical.  In  all  but  one  case,  the  analysis  indicated  that  the  terrain 
had  a  significant  effect  on  the  detection  ranges.  The  day  deliberate 
defense  was  conducted  in  an  area  where  only  minimal  terrain  modifications 
were  needed.  This  area  was  virtually  devoid  of  vegetation.  Therefore,  the 
lack  of  any  statistical  difference  between  the  modified  terrain  and  default 
terrain  detection  ranges  was  expected.  The  detection  range  distributions 
for  the  day  deliberate  defense  could  be  assumed  to  be  the  same  because  of 
their  similarities. 
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V.   CONCLUSIONS /RECOMMENDATIONS 

A.  SUMMARY  OP  FINDINGS 

The  purpose  of  this  thesis  was  to  compare  and  analyze  the  effects  of  the 
Janus (A)  default  terrain  and  the  Janus (A)  modified  terrain  on  the  simulated 
Javelin  antitank  weapon  operational  test.  The  results  suggested  that  modifying 
the  vegetation  codes  to  better  represent  the  actual  test  site  terrain  will  result 
in  a  significant  difference  in  the  detection  ranges.  This  means  that  the  lines 
of  sight  (LOS)  for  direct  fire  combat  systems  when  employed  on  the  modified 
terrain  will  be  significantly  different  than  those  employed  on  the  default 
terrain. 

This  analysis  does  not  invalidate  the  Janus (A)  model  using  the  default 
terrain.  However,  this  analyst  believes  better  results  could  be  obtained  by 
improving  the  current  terrain  database. 

B.  RECOMMENDATIONS  POR  FURTHER  RESEARCH 

The  findings  of  this  thesis  suggest  the  following  recommendations  for 
further  research  on  the  Janus (A)  terrain  database: 

•  There  seems  to  be  a  need  for  a  higher  resolution  terrain  database  to  capture 
these  terrain  features.  The  fifty  meter  resolution  does  not  provide 
accurate  enough  detail  to  represent  vegetation,  small  rolling  hills  or  other 
obstacles  that  effect  the  LOS  of  the  model. 

•  Research  needs  to  be  conducted  to  more  accurately  represent  the  size  and 
shape  of  a  particular  type  of  cultural  feature  in  a  geographic  region  (i.e., 
coniferous  trees,  deciduous  trees,  orchards,  rock  croppings,  etc).  This 
modeling  effort  should  also  look  to  represent  these  features  from  a  ground 
level  view  as  well  as  from  an  aerial  view.  The  Perspective  View  Generator 
and  Analysis  Systems  for  Unmanned  Sensors  (PEGASUS)  replicator  has  made  in- 
roads in  this  area. 
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APPENDIX  A:   PRIMARY  FACTORS 

These  are  the  primary  factors  associated  with  each  basic  terrain 
factor  used  in  the  Waterways  Experimentation  Station  (WES)  process. 

Land  use 

General  land  use 

Forest  stands 

Agricultural  crops 

Understory  density 

Urban  structures 

Embankments 
Slope 

Slope,  percent 

Azimuth,  degree 

Elevation,  meter 
Soils 

Soil  type 

Depth  of  soil 
Obstacles 

General  vertical  obstacles 

Embankments 

Roadside  ditches 
Linear  features 

Roads 

Rivers  and  streams 

Embankments 

Roadside  ditches 
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APPENDIX  B:   JANUS (A)  DEFAULT  AND  MODIFIED  SCENARIO  TERRAIN 

The  first  picture  shows  the  Janus (A)  default  terrain  for  the  Javelin 
test  area.  As  can  be  seen,  the  vegetation  areas  have  a  homogeneous  density 
over  a  particular  area.  The  second  picture  shows  the  Janus (A)  modified 
terrain.  There  is  a  marked  difference  in  the  densities  of  vegetation  areas 
compared  to  the  default  terrain.  Both  sets  of  terrain  have  a  resolution 
of  50  meters  and  a  contour  interval  of  33  feet. 
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